Soft prompts have been popularized as a cheap and easy way to improve task-specific LLM performance beyond few-shot prompts. Despite their origin as an automated prompting method, however, soft prompts and other trainable prompts remain a black-box method with no immediately interpretable connections to prompting. We create a novel theoretical framework for evaluating the interpretability of trainable prompts based on two desiderata: faithfulness and scrutability. We find that existing methods do not naturally satisfy our proposed interpretability criterion. Instead, our framework inspires a new direction of trainable prompting methods that explicitly optimizes for interpretability. To this end, we formulate and test new interpretability-oriented objective functions for two state-of-the-art prompt tuners: Hard Prompts Made Easy (PEZ) and RLPrompt. Our experiments with GPT-2 demonstrate a fundamental trade-off between interpretability and the task-performance of the trainable prompt, explicating the hardness of the soft prompt interpretability problem and revealing odd behavior that arises when one optimizes for an interpretability proxy.

Jason Wang, technical director of the National Security Agency's Computer and Analytic Sciences Research Group, said he predicts the intelligence community will need artificial intelligence to protect U.S. networks in the future. Wang said at a virtual event on July 12th that intelligence community components need to pursue more partnerships to maximize capabilities against adversaries, according to an article published by NSA. "At the NSA, with most of our industry and academic counterparts, our journey started in this area of natural language processing and computer vision -- applying capabilities like machine transcription, machine translation … to our mission," he stated. Wang said NSA has been working to mature these foundational AI applications to support core missions, including the agency's cybersecurity triage mission.